1. Field of the Invention
The present invention relates to an image pickup apparatus such as a still camera, a video camera, or the like and an image pickup method therefor and, more particularly, to an image pickup apparatus which comprises tracking means for holding an object at the center of the frame and recognizing an object position in the frame, and an image pickup method therefor.
2. Related Background Art
In the field of image pickup apparatuses such as video cameras, still cameras, and the like, recently, size and weight reductions that have been made are remarkable, and at the same time, apparatuses tend to have many functions. As one of such functions, a high zoom ratio of a photographing lens is known, and zoom lenses with the zoom ratios of .times.10, .times.12, and the like are popularly used in home-use cameras.
However, as the zoom ratio of a zoom lens increases, when the focal length at the telephoto focal point side assumes a larger numerical value, even a slight camera shake (or motion) leads to a large change in field angle, and seriously influences an image to be recorded. In an apparatus such as a video camera for performing a dynamic image photographing operation, the object may unnaturally move in the frame. Also, in a still image, a blurred image with a low resolution may be recorded. A still image can be avoided from being blurred to some extent by, e.g., increasing the shutter speed. However, in dynamic image recording, since recording is originally performed in the time base direction, the influence of a motion cannot be eliminated by only setting an appropriate shutter speed. Under these circumstances, a motion prevention apparatus for eliminating the influence of a motion has been put into practical applications mainly in the field of video cameras.
The motion prevention apparatus comprises motion detection means for detecting a motion component, and motion correction means for correcting the motion in correspondence with the detection result of the motion detection means. Of these means, as the motion detection means, a so-called electronic detection method of comparing images of two successive frames, or a method of directly measuring the actual movement of a camera using an angular velocimeter, an angular accelerometer, or the like, is known.
As the motion correction means, a so-called electronic correction method of electronically selecting an actual recording range (clipping range) from an image when an image picked up by image pickup means is read out, or optical motion correction means for optically adjusting the angle of the photographing optical axis in a direction to remove a motion is known.
Of the optical motion correction means, a method using a variable angle prism will be described below with reference to FIGS. 1A to 5. FIGS. 1A to 1C are explanatory views showing a variable angle prism (VAP). Referring to FIGS. 1A to 1C, the variable angle prism is constituted by glass plates 21 and 23, and a bellows portion 27 consisting of, e.g., polyethylene. A transparent liquid such as silicone oil is sealed in the space surrounded by the glass plates and the bellows. In FIG. 1B, the two glass plates 21 and 23 are parallel to each other, and in this case, the incident angle and exit angle of a light ray in the variable angle prism are equal to each other. On the other hand, when the two glass plates form a certain angle, as shown in FIGS. 1A and 1C, a light ray is deflected through a given angle, as indicated by light rays 24 and 26.
Therefore, when a camera is tilted due to, e.g., a motion, the angle of the variable angle prism arranged in front of a lens is controlled to deflect a light ray by an angle corresponding to the tilt angle of the camera, thereby removing the motion.
FIGS. 2A and 2B are explanatory views showing a motion removal state. Assuming that the glass plates of the variable angle prism are parallel to each other, and the photographing optical axis agrees with the center of an object in FIG. 2A, the variable angle prism is driven for a motion of a.degree. to deflect a light ray, as shown in FIG. 2B, so that the photographing optical axis can keep agreeing with the center of the object.
FIG. 3 shows the actual arrangement of a variable angle prism unit which includes a variable angle prism, an actuator for driving the prism, and an angle sensor for detecting the angle of the prism. Since actual motions appear in every directions, each of the front and rear glass surfaces of the variable angle prism is rotatable about rotation axes in two directions having a 90.degree. difference therebetween. In FIG. 3, suffixes a and b indicate components in these two rotation directions, and the components with the corresponding reference numerals have the same functions. Some b-side components are not shown in FIG. 3.
A variable angle prism main body 41 is constituted by glass plates 21 and 23, a bellows portion 27, and an inner liquid. The glass plates 21 and 23 are integrally attached to holding frames 28a and 28b using, e.g., an adhesive. The holding frames 28a and 28b constitute rotation shafts 33a and 33b together with stationary members (not shown), and are rotatable about these shafts. The shafts 33a and 33b have a 90.degree. difference therebetween. A coil 35a is integrally arranged on the holding frames 28a and 28b, and a magnet 36a and yokes 37a and 38a are arranged on a stationary portion (not shown). Therefore, when a current is supplied to the coil 35a, the variable angle prism pivots about the shaft 33a. A slit 29a is formed at the distal end of an arm portion 30a integrally extending from the holding frames 28a and 28b, and constitutes an angle sensor together with a light-emitting element 31a such as an iRED element and a light-receiving element 42a such as a PSD provided to the stationary portion.
FIG. 4 is a block diagram showing a vibration prevention lens system in which a motion prevention apparatus which comprises a variable angle prism as the motion correction means is combined with a lens system. Referring to FIG. 4, the system includes the variable angle prism 41, angle sensors 43 and 44, amplification circuits 53 and 54 for amplifying the outputs from the angle sensors, a microcomputer 45, motion detection units 46 and 47 comprising, e.g., angular accelerometers, actuators 48 and 49 constituted by the above-mentioned members from the coil 35a to the yoke 38a, and a lens 52.
The microcomputer 45 determines currents to be supplied to the actuators 48 and 49 to control the variable angle prism 41 in an angle state optimal for removing a motion in correspondence with the angle states detected by the angle sensors 43 and 44 and the detection results of the motion detection units 46 and 47. Note that the principal components in FIG. 4 constitute two blocks since the control operations in two directions having a 90.degree. difference therebetween are independently performed.
FIG. 5 is a sectional view showing the detailed structure of a conventional variable angle prism. Referring to FIG. 5, the variable angle prism is constituted by glass plates 21 and 23, a liquid 22 sealed inside the prism, a bellows portion 27, and an optical axis 25. The bellows portion 27 is formed by four doughnut-shaped films 59 to 62. The films 59 to 62 are coupled to each other by coupling portions 57, and are coupled to frame members 55 by coupling portions 58. The frame members 55 are paired with frame core members 56.
Of these portions, the coupling portions 57 between the films are formed by welding. For this reason, at least two surface layers of each of the films 59 to 62 preferably consist of a material which can provide a good heat seal, and for example, polyethylene (PE), polypropylene (PP), or the like is normally used.
The frame member 55 and the glass plate 21 or 23 are fixed using an adhesive. When the coupling portion 58 between the frame member 55 and the film 59 or 62 constituting the bellows is formed by welding as in the coupling portion 57, the same material as that of the surface of each film must be used. However, the above-mentioned materials such as PE, PP, and the like which can provide a good heat seal have lower parts precision as compared to, e.g., polycarbonate (PC) which is popularly used for such a lens barrel, and easily deform due to low rigidity. Therefore, the frame core members 56 are arranged to reinforce the frame members 55. Each frame core member 56 consists of a plastic material or a metal such as aluminum, stainless steel, or the like, which has a higher rigidity and higher thermal deformation temperature than those of the material of the frame member 55. The frame member 55 is formed around the frame core member 56 by, e.g., insert molding with reference to the frame core member 56. This structure can assure high flatness of the film welded portion of the frame member 55, high mechanical strength or dimensional precision of the support portion of the glass plate 21 or 23, and high dimensional precision of the fitting diameter of glass.
In the above description, a variable angle prism is used as the optical motion correction means. As another example of the optical motion correction means, a method of movably disposing a correction optical system disclosed in, e.g., U.S. Pat. No. 2,959,088 will be described below. FIG. 6 is a sectional view showing the arrangement of the overall optical system of the optical motion correction means. Referring to FIG. 6, lenses 71 and 72 serve as a correction optical system for main lenses 82 and 83. The focal lengths of the correction optical system are set as follows. Let f1 be the focal length of the lens 71 having negative power, and f2 be the focal length of the lens 72 supported by a movable support portion 73 and having positive power. The focal lengths of the lenses are set to satisfy the relation f1=-f2.
Furthermore, the lens 72 is supported by a gimbals 75 to realize a biaxial movable support mechanism. In order to keep balance with the correction optical system, a counterweight 80 is provided.
When such an optical condition is satisfied, a motion prevention apparatus including so-called inertial pendulum type optical motion correction means can be realized.
The biaxial movable mechanism of the gimbals 75 will be described below. FIG. 7 is a sectional view showing the structure of principal part of the optical system of the optical motion correction means. The lens 72 is supported by a support member 75y which has a degree of freedom in the y-axis direction, and the support member 75y is supported by a support member 75x having a degree of freedom in the x-axis direction perpendicular to the y-axis direction. Furthermore, the support member 75x is supported by a lens barrel 74.
With this arrangement, a correction optical system with degrees of freedom in two axis directions can be realized.
A typical zoom lens system upon combination of the motion correction means having the above-mentioned variable angle prism and a zoom lens will be exemplified below. In this case, an inner or rear focus type zoom lens system which attains focusing by a lens group behind a variator lens group for zooming will be exemplified.
Various lens systems of such a lens type are known. In this case, a lens arrangement which uses the rearmost lens group for focusing will be exemplified. FIG. 8 is a sectional view showing the arrangement of the lens system. Referring to FIG. 8, the lens system includes a stationary front-element lens group 111, a variator lens group 112, a stationary lens group 113, and a focus (compensator) lens group 114. The lens system also includes an anti-rotation guide rod 133, a variator feed rod 134, a stationary lens barrel 135, an iris unit 136 (which is inserted in a direction perpendicular to the plane of the drawing of FIG. 8), a stepping motor 137 serving as a focus motor, and an output shaft 138 of the stepping motor. A male screw is formed on the output shaft 138 so as to move the lens. A female screw portion 139 meshes with this male screw, and is integrated with a movable frame 140 for the lens group 114. The lens system further includes guide rods 141 and 142 for the movable frame of the lens group 114, a rear plate 143 for aligning and pressing the guide rods, and a relay holder 144. The lens system also includes a zoom motor 145, a deceleration unit 146, and interlocking gears 147 and 148. The interlocking gear 148 is fixed to the zoom (variator) feed rod 134.
With the above-mentioned arrangement, when the stepping motor 137 is driven, the focus lens group 114 moves in the optical axis direction by a screw feed mechanism. When the zoom motor 145 is driven, the gears 147 and 148 are interlocked with each other, and the rod 134 is rotated, thus moving the variator lens group 112 in the optical axis direction.
The positional relationship between the variator lens and the focus lens in the above-mentioned lens system will be described in correspondence with some distances. FIG. 9 is a graph showing the relationship between the variator position and the focus lens position. In this case, FIG. 9 shows the in-focus positional relationships for objects separated by distances of infinity .infin., 2 m, 1 m, 80 cm, and 0 cm. In the inner focus lens system, since the positional relationship between the variator and focus lenses varies depending on the object distance, the lens groups cannot be interlocked by a simple mechanical structure like a cam ring of a front-element focus lens system.
Therefore, if the zoom motor 145 is simply driven in the structure shown in FIG. 8, an out-of-focus state occurs. Since the inner focus lens system has the above-mentioned characteristics, its practical application is not easily attained although it has an advantage of "a small number of constituting lenses" and the like in addition to the above-mentioned advantage "good closest-distance photographing performance".
However, in recent years, a control technique of attaining an optical lens positional relationship shown in FIG. 9 in correspondence with the object distance has been developed, and is applied to commercial products. For example, Japanese Laid-Open Patent Application Nos. 1-280709 (U.S. Pat. No. 4,920,369), 1-321416, and 2-144509 proposed by the present applicant each disclose a method of tracking a locus representing the positional relationship between the two lenses in correspondence with the distance.
In Japanese Laid-Open Patent Application No. 1-280709, the positional relationship between the variator and compensator (focus lens) is maintained by a method shown in FIGS. 10 to 12. FIG. 10 is a block diagram of a lens control circuit corresponding to FIGS. 8 and 9. Referring to FIG. 10, lens groups 111 to 114 are the same as those shown in FIG. 8. The position of the variator lens group 112 is detected by a zoom encoder 149. Note that the encoder may comprise a volume encoder in which a brush integrally attached to a variable movable ring slides along a circuit board printed with a resistor pattern. An iris encoder 150 detects the iris value, and uses, e.g., an output from a Hall element in an iris meter. An image pickup element 151 such as a CCD is connected to a camera processing circuit 152. A Y (luminance) signal output from the camera processing circuit 152 is supplied to an AF circuit 153. The AF circuit 153 discriminates the in-focus or out-of-focus state. In the case of the out-of-focus state, the circuit 153 discriminates the near or far focus state and the defocus amount. These discrimination results are supplied to a CPU 154. A power-ON reset circuit 155 performs various reset operations when the power switch is turned on. A zoom operation circuit 156 supplies the operation contents of a zoom switch 157 by an operator to the CPU 154. A memory unit of locus data shown in FIG. 8 stores direction data 158, speed data 159, and boundary data 160. A zoom motor driver 161 and a stepping motor driver 162 are connected to the CPU 154. The number of input pulses from the stepping motor is continuously counted by the CPU 154, and the count value is used as an encoder of the absolute position of the focus lens. In this arrangement, since the variator position and the focus lens position are respectively determined by the zoom encoder 149 and the number of input pulses from the stepping motor, one point on the map shown in FIG. 9 is determined. On the other hand, the map shown in FIG. 9 is divided into small strip-shaped regions on the basis of the boundary data 160. FIG. 11 is an explanatory view showing the relationship between the variator position and the focus lens position, which is divided into strip-shaped regions. In FIG. 11, a hatched portion corresponds to a region where the lenses are inhibited from being disposed. When one point on the map is determined in this manner, a small region to which the point belongs can be determined.
The speed and direction data are stored in units of regions as the rotational speeds and directions of the stepping motor, which are calculated on the basis of the locus passing the centers of the regions. For example, FIG. 11 is divided into 10 zones. Assuming that the zoom time requires 10 sec, the passing time per zone is 1 sec, as a matter of course. FIG. 12 is an enlarged explanatory view of block III. A locus 164 passes the center of block III, a locus 165 passes the lower left corner of block III, and a locus 166 passes the upper right corner of block III. Note that the central locus can be tracked without causing any error if the lens moves at a speed of x mm/sec.
If the speed calculated in this manner is called a region representative speed, the speed memory stores values in correspondence with the number of small regions. If this speed is represented by reference numeral 168, the representative speed is finely adjusted like a speed 167 or 169 in correspondence with the detection result of an auto focus detection device, thereby setting the stepping motor speed. In addition, since the rotation direction of the stepping motor changes depending on a region even in a zoom operation from the telephoto end to the wide end (or vice versa), a sign indicated by the direction data is stored.
As described above, when the focus lens position is controlled by driving the stepping motor in the zoom driving operation using the stepping motor speed which is determined by correcting the region representative speed calculated based on the variator position and the focus lens position is corrected on the basis of the detection result of the auto focus detection device, an out-of-focus state during the zoom driving operation can be prevented even in the inner focus lens system.
Note that the following method is also proposed (U.S. Pat. No. 5,005,956). In this method, the speeds indicated by the arrows 167, 169, and the like are stored in units of blocks in addition to the representative speed indicated by the arrow 168, and one of the three speeds is selected in correspondence with the detection result of the auto focus detection device.
In place of the method of storing the speeds, the following method is known. In this method, some curves representing the focus lens positions corresponding to a plurality of variator positions are stored in correspondence with the object distances of .infin., 2 m, 1 m, and the like shown in FIG. 9, and when the object distance is an intermediate distance between the stored curves, the positional relationship to be set of the two lens groups is interpolated using the data of the stored curves. FIG. 13 is a sectional view showing the arrangement of the motion correction means obtained when the variable angle prism is coupled to a zoom lens system. Referring to FIG. 13, a rotation shaft portion 263 is arranged integrally with a holding frame 28, and a rotation shaft 267, at the side opposite to the rotation shaft 263, is not integrally arranged with the holding frame 28 but is constituted by press-fitting a shaft consisting of a metal such as stainless steel into the holding frame. A leaf spring 268 is fixed by a screw 269. A flat glass 266 is arranged to avoid, e.g., a photographer from directly touching and damaging the variable angle prism. A mounting screw 265 is used for mounting an accessory. A stationary lens barrel member 264 includes holes for receiving the rotation shafts of the variable angle prism.
The lens barrel member 264 is fastened to a stationary lens barrel 135 of the zoom lens system by screws 270. In FIG. 13, a holding frame for pivoting the front-side glass of the variable angle prism, actuators, and sensor members for detecting the angles are not shown for the sake of simplicity.
The examples using the variable angle prism and the movable correction optical system as the optical blur correction means, and the zoom lens systems combined with these optical motion correction means have been described.
A prior art associated with so-called electronic means as the motion detection means will be briefly described below. FIG. 14 is a block diagram showing the arrangement of the electronic motion detection means. This motion detection means is described in Japanese Laid-Open Patent Application No. 2-75283 (U.S. Pat. No. 5,107,293). An image pickup signal from a CCD is amplified by a preamplifier 313, and the amplified signal is supplied to a motion detection circuit 351. In this circuit, the frame is divided into a plurality of blocks, feature points are extracted in units of blocks, and the feature points are compared with those of a temporally different field frame, thereby calculating motion vectors in units of blocks. A motion vector memory 352 stores the motion vectors in units of blocks on the frame, which are calculated by the motion detection circuit 351, for one frame. A representative vector operation circuit 353 operates and synthesizes the motion vector data of the respective blocks stored in the memory 352 in accordance with a predetermined algorithm, and outputs a representative motion vector V for one frame. With the above-mentioned basic arrangement, the direction of a motion can be obtained as the representative vector.
Note that FIG. 14 also includes peripheral circuits such as a circuit 354 for weighting the respective blocks upon calculation of the representative vector, a mode input circuit 315 used for the purpose of varying the weighting contents in correspondence with a mode, and the like.
In the above-mentioned motion prevention apparatus, in order to remove an unnecessary motion, the motion correction means is driven in correspondence with the detection result of the motion detection means, thereby removing a motion. However, in practice, in the case of an apparatus such as a video camera for photographing dynamic images, if the motion correction means faithfully responds to, e.g., a panning operation that a photographer intended, another problem is posed. In order to solve this problem, it is checked based on the frequency, amplitude, and the like of the detected motion if the motion is to be corrected by the motion correction means, and if it is determined that the motion is caused by a panning operation, the motion correction means is inhibited from responding to the motion.
For this reason, when a photographer performs a photographing operation without looking into the finder, and an object gradually deviates from the center of the frame, if the motion prevention apparatus determines this motion as a panning operation, the object may deviate from the center of the frame and fall outside the frame. In addition to such a specific condition, if the motion correction means is inhibited from responding to frequency components (mainly, low frequency components), determined as a panning operation, of those of a motion (in an actual arrangement, for example, the output from the motion detection means is supplied to a low-cut filter), only a vibration of low frequency components remains. As a result, only a slow vibration may remain on the frame. Furthermore, when the motion detection means comprises a sensor such as an acceleration sensor, if the sensor does not have a sufficient detection frequency range, a similar phenomenon may occur.
One of the arrangements for solving these problems is disclosed in Japanese Laid-Open Patent Application No. 5-304631 proposed by the present applicant. The position, in the frame, of a principal object, which cannot be detected by the conventional motion detection means is specified by detecting the viewpoint that indicates the viewpoint position, in the finder, of a photographer. Based on this position information, a photographing operation according to the photographer's intention is attained. For example, in the above-mentioned disclosure, when an object is specified by viewpoint detection means, the detection region of the above-mentioned electronic motion detection means is limited to a region other than the object (the motion vectors in units of blocks on an object are not used). With this control, in a recorded image, the movement of the background image is inhibited, and only a natural movement of the object is permitted. In other words, an image free from a motion can be obtained as if it were photographed by a camera fixed on a tripod.
However, in the prior art, no countermeasures are taken against the following problems, and further improvements are demanded.
(a) When the viewpoint detection means recognizes an object, if the viewpoint position is always determined as the object position, an operation error may occur when the viewpoint gazes at a position other than the object. PA1 (b) It is difficult to hold an object within the field angle when a hand-held no-finder photographing operation is performed for a large focal length. PA1 (c) It is difficult to remove unnecessary motion components and to obtain a high-quality recorded image when a moving object to be photographed is tracked by a panning operation and is kept held at the center of the frame. PA1 (d) No proposal associated with a technique for holding an object within a frame is made.
Note that as the prior applications of a tracking apparatus by the present applicant, U.S. Pat. Nos. 5,031,049 and 4,872,058 are known, and as the prior application of the vibration prevention apparatus, U.S. Pat. No. 5,012,270 or the like is known.